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PREFACE 
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1.  INTRODUCTION  AND  BACKGROUND 


The  complete  oil  breakdown  rate  analyzer1'3,  version  II  (COBRA  II)  is  being 
fielded  at  most  US  Air  Force  and  some  Air  National  Guard  bases  which  have  F-15  or  F- 
16  aircraft  powered  by  the  Pratt& Whitney  FI 00  turbine  engine.  The  COBRA  II  is  a  new 
model  of  an  instrument  which  monitors  the  thermal  degradation  level  of  synthetic  ester- 
based  turbine  engine  lubricants,  such  as  MIL-L-7808  and  MIL-L-23699  (now  MIL-PRF- 
7808  &  23699)  oils.  The  instrument  works  by  measuring  the  relative  electrical 
conductivity  of  a  thin  film  of  oil.  The  conductivity  of  an  ester  lubricant  increases  with 
thermal  degradation,  and  COBRA  II  takes  advantage  of  this  phenomenon  by  increasing  in 
reading  with  increase  in  conductivity  of  the  oil.  The  instruments  cost  about  $4,500  each. 
Fielding  of  the  COBRA  IIs  for  FI  00  engine  oil  analysis  was  necessitated  by  a  number  of 
engine  oil  thermal  breakdowns  due  to  bearing  compartment  over-temperature  events, 
colloquially  known  as  the  FI 00  “black  oil  problem”.  Recently,  trending  and  absolute 
limits  of  COBRA  II  readings  have  been  established  for  the  FI 00  engine4,  and  a  number 
of  these  engines  have  been  “saved”  by  appropriate  maintenance  actions  instituted  after 
COBRA  II  analysis  indicated  an  impending  black  oil  problem. 

Although  COBRA  II  is  not  recommended  for  use  in  monitoring  programs  for 
most  turbine  engines  which  have  very  little  tendency  to  significantly  thermally  degrade 
the  lubricant,  it  can  enhance  safety  and  be  quite  cost-effective  when  applied  to  analysis  of 
oil  from  engines  which  have  a  history  of  thermal  degradation  problems.  Use  of  COBRA 
II  and  its  predecessor,  COBRA,  have  been  successful  in  detecting  oil  degradation 
problems  and  saving  millions  of  dollars  in  the  TF34  and  FI  00  engine  programs. 

The  original  COBRA  instruments,  which  were  analog  electronic  devices,  were 
fielded  at  most  A-10  (TF34  engine)  bases  in  the  early  1980s  due  to  an  oil  thermal 
degradation  problem  which  resulted  in  engine  mainshaft  fracture.  These  COBRAs  were 
highly  successful  in  trending  the  oil  and  detecting  impending  failures  in  the  TF34.  The 
TF34’s  oil  problem  was  eventually  traced  to  a  poorly  designed  oil  sump  seal,  which 
allowed  hot  gas  path  air  into  the  oil  compartment.  The  seal  was  redesigned  and  the  oil 
problem  ceased  after  retrofit.  The  COBRAs  were  then  taken  out  of  service  circa  1985, 
when  their  use  was  no  longer  cost-effective. 

In  the  early  1990s,  the  F100  engine  began  to  experience  No.  5  bearing 
compartment  over-temperature  events,  which  led  to  oil  thermal  degradation,  and  came  to 
be  known  as  the  FI 00  black  oil  problem.  In  1993  COBRA  was  upgraded  with  digital 
electronics  and  fielded  at  several  FI  00  bases  for  testing  of  its  ability  to  trend  FI  00  oil  and 
detect  impending  distress  due  to  black  oil.  Although  the  testing  was  partially  successful, 
with  at  least  six  instances  of  COBRA  detecting  an  impending  problem,  the  FI  00  program 
office  at  Kelly  AFB  TX  felt  that  maintenance  personnel  could  sufficiently  detect  black  oil 
just  as  well  as  COBRA  by  using  their  eyes  and  nose  to  determine  the  dark  coloration  and 
burned  smell  of  an  abnormal  oil  sample.  They  also  felt  that  COBRA  had  failed  to  prove 
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that  it  could  adequately  trend  the  problem.  Therefore,  COBRA  was  withdrawn  from  the 
field  in  early  1994. 

However,  the  black  oil  problem  continued.  Sight  and  smell,  being  highly 
subjective,  were  proving  to  be  very  uncertain  identification  methods  and  many  black  oil 
incidents  escaped  detection.  Meanwhile,  scientists  and  engineers  from  the  Air  Force  and 
Pratt  &  Whitney  realized  that  COBRA  still  had  excellent  potential  to  detect  this  problem, 
but  needed  further  improvement.  This  was  especially  true  with  respect  to  the  complaint 
by  field  personnel  in  the  past  that  there  was  too  much  “drift”  in  the  readings.  By  this, 
they  meant  that  sometimes  the  COBRA  reading  would  tend  to  creep  higher  as  long  as  the 
one  kept  the  “On”  button  depressed.  It  was  often  difficult  to  make  a  call  on  what  the 
actual  reading  should  be.  This  was  a  legitimate  concern  that  was  relayed  to  the  COBRA 
manufacturer,  NAECO  Associates,  Inc.,  Arlington  VA.  To  address  the  concern,  the 
author  worked  with  the  COBRA  inventor  and  lead  NAECO  engineer  in  1995  to  diagnose 
the  cause  of  the  drift  problem.  It  was  traced  to  a  key  resistor  in  the  analysis/readout 
electrical  circuit  that  sometimes  allowed  back  leakage  of  current  into  the  readout  module. 
This  back  leakage  caused  the  liquid  crystal  display  (LCD)  readout  to  increase  as  one 
continued  to  depress  the  On  button.  The  simple  solution  to  the  problem  was  replacement 
of  the  culprit  resistor  with  a  similar  resistance  diode.  The  diode  allows  current  flow  in 
only  one  direction,  thereby  eliminating  the  potential  for  back  leakage.  After  the  diode 
was  installed,  the  drift  problem  was  minimized  and  confidence  in  the  readings  has 
improved. 

In  1996,  some  bases  which  had  experienced  a  number  of  black  oil  incidents 
purchased  COBRAs  to  supplement  their  sight/smell  detection  method.  Several  engines 
were  saved  by  corrective  maintenance  actions  taken  after  COBRA  indicated  an 
impending  black  oil  failure.  Tragically  in  1996,  there  were  two  aircraft  lost  due  to  the 
black  oil  problem,  because  maintenance  personnel  were  unsure  if  the  oil  was  dark  enough 
or  smelled  burned  enough  to  make  a  black  oil  call  .  The  Air  Force  subsequently  decided 
that  reliance  on  subjective  human  senses  was  no  longer  an  adequate  defense  against  this 
problem.  Too  many  black  oil  events  had  escaped  detection  by  maintenance  personnel, 
and  an  objective  diagnostic  method  was  sought.  COBRA  was  chosen  as  the  best 
available  technique  for  detection  of  this  particular  problem.  It  was  decided  to  change  the 
name  to  COBRA  II,  to  make  clear  this  was  a  new  version  of  the  instrument.  The  Air 
Force  began  purchasing  and  fielding  COBRA  IIs  on  a  widespread  basis  in  April  1997. 
They  are  now  in  service  at  most  USAF  FI 00  bases.  The  Air  National  Guard  also  has 
COBRA  II  at  many  of  its  bases. 

The  oil  problem  in  the  FI 00  engine  has  been  traced  to  overheating  of  the  No.  5 
(and  in  rare  cases,  the  No.  4)  bearing  compartment,  which  is  located  in  the  low  pressure 
turbine  (LPT)  module.  An  Air  Force/Pratt  &  Whitney  integrated  product  team  (IPT) 
studied  the  problem  in  1993-94  and  recommended  a  redesign  of  the  No.  5  compartment, 
which  included  providing  additional  heat  shielding  to  the  bearing’s  oil  supply  and 
scavenge  tubes.  Unfortunately,  due  to  funding  shortfalls,  the  IPT  recommendations  for 
retrofit  were  not  implemented  until  1997.  The  new  components  are  expected  to  alleviate 
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the  black  oil  problem,  analogous  to  the  success  of  the  TF34  oil  sump  seal  retrofit  of  the 
last  decade.  Until  the  retrofit  is  complete  and  proven  to  stop  the  black  oil  problem, 
however,  COBRA  II  will  be  an  important  part  of  the  FI  00  engine  condition  monitoring 
program.  As  of  September  1997,  approximately  60%  of  USAF’s  F100  engines  had 
undergone  the  No.  5  retrofit. 

Pratt  &  Whitney  submitted  a  component  improvement  program  (CEP)  proposal  in 
early  1997  for  a  field  service  evaluation  (FSE)  to  track  and  analyze  the  data  from 
COBRA  II  over  its  first  months  in  operation6.  This  CEP  was  approved  in  March  1997, 
and  ran  until  October.  It  includes  recommendations  and  lessons  learned  on  the  field 
performance  of  COBRA  EL  As  a  part  of  this  program,  six  lead  bases  plus  Wright 
Laboratory  (WL)  at  Wright-Patterson  AFB  OH  were  requested  to  participate  in  a 
correlation  study  of  COBRA  II  performance  with  respect  to  repeatability  and 
reproducibility  of  readings.  Since  the  author  has  extensive  experience  with  COBRA  and 
COBRA  II,  he  was  asked  to  make  up  the  samples  and  conduct  the  data  analysis  for  the 
correlation  study.  The  following  sections  of  this  report  detail  the  study  findings  and 
summarize  the  field  performance  to  date  for  COBRA  II. 
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2.  EXPERIMENTAL 


The  COBRA  II  is  a  relatively  simple  instrument  which  measures  the  ability  of  a 
thin  film  of  oil  to  conduct  electricity.  When  synthetic  ester-based  lubricants  undergo 
significant  thermal  stress,  acids,  alcohols,  and  other  electrical  charge  carrier  species  are 
formed  which  increase  the  conductivity  of  the  oil.  COBRA  II  readings  increase  with 
increase  in  conductivity  of  a  fluid,  thereby  providing  a  means  to  monitor  the  relative  level 
of  thermal  degradation  within  an  ester  lubricant.  A  picture  of  the  instrument  is  shown  in 
Figure  1.  The  COBRA  II  measures  10  x  5  x  5  inches,  about  the  size  of  a  child’s  lunch 
box.  It  weighs  just  under  7  pounds. 


Zero  knob 


LCD  Readout 


On  button 


Calibration  knob 


Electrodes 


*  t  . 


|i||p 

3  %  -a-  '■  L 


Glass  Rod  Calibration  Fluid 


Figure  1.  COBRA  II  Oil  Analyzer 
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In  application,  a  glass  rod  or  pipette  is  dipped  into  the  sample  and  one  or  two 
drops  (-0.025  -  0.05  ml)  of  oil  are  placed  on  the  bottom  electrode  disk.  The  rod  or 
pipette  is  then  used  to  “spread”  the  oil  so  that  the  bottom  electrode  is  completely  covered, 
forming  a  thin  oil  film.  The  hinged,  top  electrode  is  then  brought  down  to  mate  with  the 
bottom  electrode,  creating  an  oil  film  “sandwich”  between  the  two  electrodes.  The  “On” 
button  is  then  depressed  and  held  until  a  stable  reading  appears  on  the  LCD,  which 
usually  takes  about  2  seconds  with  COBRA  II.  In  the  older  model  COBRA,  stabilization 
sometimes  took  over  4  seconds  or  did  not  occur  at  all,  i.e.,  the  reading  would  continue  to 
drift  higher  and  higher  as  long  as  the  On  button  was  held.  Field  personnel  are  instructed 
to  take  the  reading  at  4  seconds,  if  stabilization  does  not  occur  before  that  time.  After 
each  analysis,  the  electrodes  are  cleaned  with  a  Kimwipe™  and  solvent  before  the  next 
sample  is  run.  It  usually  takes  15-30  seconds  between  COBRA  II  analyses,  depending  on 
the  operator.  The  procedure  is  extremely  simple  for  a  laboratory  instrument.  Most 
operators  can  be  trained  in  a  few  minutes  and  become  proficient  shortly  thereafter. 

COBRA  II  readings  are  integer,  dimensionless  numbers.  It  has  been  suggested  to 
the  manufacturer  that  the  readings  be  related  to  the  SI  unit  for  electrical  conductance, 
known  as  Siemens  (S).  One  Siemens  is  equal  to  one  Ampere  divided  by  one  Volt  (S  = 
A/V).  The  suggestion  has  been  taken  under  consideration. 

The  instruments  are  calibrated  daily  with  an  ester-based  fluid  which  gives  a 
COBRA  II  reading  of  8.  New  ester  oils  usually  read  0  to  2  on  COBRA  II.  Used  oil 
samples  from  normally  operating  (no  black  oil  problem)  F100  engines  typically  read  1-4. 
An  increase  in  reading  of  3  or  more  within  10  hours  of  operation  usually  signals  that  an 
impending  black  oil  problem  is  developing.  A  reading  of  10  or  more  has  so  far  always 
been  indicative  of  impending  or  actual  mechanical  distress.  These  trending  and  absolute 
limits  are  considered  preliminary,  and  will  be  modified  as  appropriate  during  the  current 
CIP.  Samples  from  engines  which  have  had  documented  compartment  over-temperature 
events  have  read  anywhere  from  10  to  above  60,  depending  on  the  severity  of  the 
situation. 

Non-destructive  inspection  (NDI)  laboratories  at  six  lead  operational  bases  and 
the  Lubrication  Branch  of  Wright  Laboratory  (WL/POSL)  at  Wright-Patterson  AFB  were 
asked  to  participate  in  this  field  performance  &  correlation  study.  Several  of  the  bases 
had  2  COBRA  II  units  in  operation,  and  these  were  included  in  the  study.  In  all,  9 
COBRA  II  instruments  were  evaluated,  making  this  a  statistically  significant  database, 
beyond  the  99%  confidence  level.  Each  base  received  3  samples,  which  gave  COBRA  II 
readings  representative  of  various  levels  of  thermal  degradation.  The  operators  were 
instructed  to  run  the  samples  5  times  each,  in  random  order. 

Since  thermally  degraded  ester  lubricants  continue  to  decompose  and  increase  in 
conductivity,  even  when  stored  at  room  temperature,  more  stable,  simulated  samples  were 
devised  for  the  correlation  study.  These  simulated  samples  were  made  from  a  typical 
ester  lubricant  basestock  compound,  trimethylolpropane  triheptanoate  (TMPH),  with  an 


5 


r 


electrically  conductive  fluid  added.  The  TMPH  molecule  gives  a  zero  reading  on 
COBRA  II.  The  conductive  fluid  is  sold  under  the  trade  name  Conostan™  Stabilizer, 
produced  by  Conoco  Inc.,  Houston  TX.  It  is  used  as  a  stabilizer  in  organometallic 
standards  and  contains  85-95%  alkylamine  alkylaryl  sulfonate  as  the  active  (conductive) 
ingredient,  with  the  remainder  being  white  mineral  and/or  solvent  neutral  oils.  The 
Conostan™  Stabilizer  was  added  to  the  TMPH  ester  in  0.22, 0.75,  and  1.4%  (by  weight) 
amounts  in  samples  1, 2,  and  3,  respectively.  These  were  made  to  simulate  the  electrical 
conductivities  characteristic  of  normal,  questionable,  and  abnormal  FI  00  oil  samples. 
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3.  DATA  ANALYSIS 


Data  were  analyzed  for  repeatability  and  reproducibility  by  use  of  simple  root 
mean  square  (RMS)  equations.  Repeatability  is  defined  as  the  measure  of  data  spread 
from  a  single  instrument,  i.e.,  it  is  the  measure  of  how  well  one  instrument  can  repeat  its 
own  results.  Reproducibility  is  defined  as  the  measure  of  agreement  of  data  from  one 
instrument  with  that  of  other  instruments.  The  RMS  equations  are  modeled  after  those 
used  in  an  earlier  study  of  COBRA  field  performance3,  but  are  shown  again  here  for 
clarity. 


Repeatability  of  each  lab’s  results  is  found  by  first  establishing  the  RMS  value  of 
the  sample  means  measured  by  that  lab. 

L  =  (2  Lj2)172 


where:  L  =  lab  RMS  sample  means  value 
Lj  =  lab  mean  for  sample  i 

Next,  a  lab  RMS  deviation  value  is  found  from  the  RMS  of  that  lab’s  sample 
standard  deviations. 

S  =  (2  Sj2) 1/2 


where:  S  =  lab  RMS  deviation  value 

Sj  =  lab  standard  deviation  for  sample  i 

The  normalized  lab  repeatability,  in  percent,  is  then  determined  by  dividing  the 
lab  RMS  deviation  value  by  that  lab’s  RMS  sample  means  value  and  multiplying  by 
100%. 


rL  =  100%(S/L) 

where:  rL  =  lab  repeatability  value  (in  percent) 

To  measure  reproducibility,  the  RMS  value  of  the  overall  sample  means  is  found. 
R  =  (2  R2)172 

where:  R  =  overall  RMS  reproducibility  value 
Ri  =  overall  mean  for  sample  i 
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Each  lab’s  RMS  reproducibility,  in  percent,  is  then  determined  by  the  normalized 
difference  between  the  lab  RMS  sample  means  value  and  overall  RMS  reproducibility 
value,  multiplied  by  100%. 

Rl  =  100%(L-R)/R 

where:  Rl  =  lab  reproducibility  value  (in  percent) 

To  determine  the  overall  mean  RMS  reproducibility,  the  overall  RMS  standard 
deviation  value  is  calculated. 

S0  =  (ISoi2)1/2 

where:  S0  =  overall  RMS  standard  deviation  value 
Soi  =  overall  standard  deviation  for  sample  i 

The  overall  mean  RMS  reproducibility  is  then  found  by  dividing  the  RMS 
standard  deviation  value  by  the  overall  RMS  reproducibility  value,  multiplied  by  100%. 

R0  =  100%(So/R) 

where:  R^,  =  overall  mean  RMS  reproducibility  value  (in  percent) 

The  lower  the  repeatability  or  reproducibility  value  (absolute  value),  the  better. 
That  is,  a  value  of  zero  represents  perfect  repeatability  or  exact  agreement  with  the 
overall  RMS  reproducibility  value,  R. 
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4.  RESULTS  AND  DISCUSSION 


Reduced  data  from  all  test  sites  are  shown  in  Table  1.  All  data  are  included  in 
these  results.  None  were  rejected  or  treated  as  outliers.  Table  2  gives  the  RMS  values 
found  for  this  data  set. 

TABLE  1.  Reduced  COBRA  II  Data 


COBRA  II  Readings 


Location 

Sample  1 

Sample  2 

SampleJ. 

Seymour  Johnson 

3±0 

7±0 

13±0 

Elmendorf 

3.4±0.55 

9±0.71 

15.6±0.55 

Nellis  1 

3±0 

9±0 

16.2±0.55 

Nellis  2 

2.8±0.45 

8.2±0.45 

15±0 

Wright-Patterson 

3±0 

9±0 

15±0 

Eglin 

2.6±0.55 

8.6±0.55 

15.6±0.55 

Langley  1 

3±0 

8±0 

13.8±0.45 

Langley  2 

3±0 

8±0 

15±0.71 

Luke 

3±0 

9.4±0.55 

17.2±0.45 

Overall  Mean* 

2.98±0.34 

8.47±0.79 

15.2±1.27 

*  These  values  are  calculated  from  the  entire  data  set.  See  summary  statistics  in 
Appendix. 
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TABLE  2.  COBRA  II  RMS  Values 


Location 

£ 

L 

rL(%) 

Bl(%) 

Seymour  Johnson 

0 

15.1 

0 

-14.7 

Elmendorf 

1.05 

18.3 

5.74 

3.39 

Nellis  1 

0.55 

18.8 

2.93 

6.21 

Nellis  2 

0.64 

17.3 

3.70 

-2.26 

Wright-Patterson 

0 

17.8 

0 

0.56 

Eglin 

0.95 

18.0 

5.28 

1.69 

Langley  1 

0.45 

16.2 

2.78 

-8.47 

Langley  2 

0.71 

17.3 

4.10 

-2.26 

Luke 

0.71 

19.8 

3.59 

11.9 

Average  Value* 

0.56 

17.6 

3.12 

5.72 

Overall  Mean** 

1.53 

17.7 

8.64 

*  These  are  merely  the  average  values  of  the  data  columns.  The  average  reproducibility 
is  found  from  the  absolute  values  of  the  lab  reproducibilities. 

**  These  are  the  overall  RMS  values,  S0,  R,  and  R0,  respectively. 

Correlation  Sample  Results 

Figure  2  graphs  the  sample  means,  with  standard  deviation  bars,  for  all  base  labs. 
The  approximate  95%  confidence  intervals  are  also  shown  for  each  sample. 

Sample  1  has  a  conductivity  level  giving  COBRA  II  readings  characteristic  of  oils 
from  normally  operating  FI  00  engines.  The  overall  mean  value  of  the  45  readings  from 
all  bases  was  2.98±0.34,  giving  a  relative  standard  deviation  (RSD)  of  1 1 .4%.  The 
lowest  lab  average  was  2.6  and  the  highest  was  3.4.  The  lowest  single  reading  was  2  (3 
times)  and  the  highest  was  4  (2  times).  The  data  range  is  only  2,  giving  confidence  that 
no  base  would  ever  mistake  this  “good”  oil  sample  for  a  questionable  or  abnormal  black 
oil  problem. 

Sample  2  is  at  a  conductivity  level  which  is  indicative  of  a  questionable  FI  00  oil 
sample.  That  is,  the  level  of  thermal  degradation  would  probably  be  to  the  point  where 
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Figure  2.  COBRA  II  Sample  Means  for  all  Base  Labs 

no  bearing  compartment  over-temperature  problem  had  yet  surfaced,  but  an  inspection  of 
the  No.  5  and/or  No.  4  bearing  compartments  for  excessive  thermal  deposition  (coking)  is 
warranted.  The  overall  mean  value  for  this  sample  was  8.47±0.79,  with  RSD  of  9.3%. 
The  lowest  reading  was  7  (5  times)  and  highest  was  10  (3  times).  The  lowest  lab  average 
was  7.0  and  the  highest  was  9.4.  There  were  17  readings  of  8  and  20  readings  of  9.  Even 
the  base  lab  at  Seymour  Johnson,  which  had  all  five  of  the  lowest  readings,  should  be 
able  to  call  for  bearing  compartment  inspection  with  high  confidence  that  this  is  a 
necessary  action,  based  on  the  deviation  in  reading  from  a  “normal”  oil  sample,  such  as 
sample  1. 

Sample  3  is  representative  of  abnormal  oil  from  an  FI  00  engine  in  which  bearing 
compartment  over-temperature  has  already  begun.  The  overall  mean  for  sample  3  was 
15.2±1.27,  with  RSD  of  8.3%.  The  lowest  reading  was  13  (6  times)  and  highest  was  18 
(once).  The  lowest  lab  average  was  13.0  and  the  highest  was  17.2.  There  were  5 
readings  of  14, 17  readings  of  15, 10  readings  of  16,  and  6  readings  of  17.  All  bases 
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would  have  easily  identified  this  as  a  definite  problem  sample,  given  the  current  limit  of 
10  as  a  reading  value  at  which  maintenance  action  must  be  taken. 

The  summary  statistics  listed  in  the  Appendix  show  the  average,  median,  mode, 
and  geometric  mean  for  each  sample  are  very  close  in  value,  indicating  the  data  have  a 
strong  central  tendency.  The  variance,  standard  deviation,  standard  error,  range,  and 
interquartile  values  illustrate  the  spread  or  dispersion  of  data  is  quite  tight.  The  low 
skewness  and  kurtosis  values  denote  that  data  distribution  for  each  sample  is  very  close  to 
Gaussian  normal,  although  the  kurtosis  coefficient  of  sample  1  is  a  relatively  large  value 
of +6.8.  This  means  the  data  distribution  for  sample  1  is  very  steep  at  the  center,  which 
is  due  to  the  fact  that  40  of  the  45  values  are  readings  of  3  (see  frequency  histogram  in 
Appendix).  Skewness  and  kurtosis  values  within  a  range  of -2  to  +2  for  large  data  sets 
indicate  there  is  not  significant  deviation  from  Gaussian  normal  distribution.  The 
coefficients  of  variation,  usually  referred  to  as  the  RSDs,  average  less  than  10%.  This 
normalized  measure  of  data  spread  reinforces  the  sense  of  centralized  distribution. 

Repeatability  and  Reproducibility 

The  average  lab  repeatability  value  was  found  to  be  3.12%,  while  the  average  lab 
reproducibility  value  was  5.72%.  These  values  for  COBRA  II  are  14%  and  18%  better, 
respectively,  than  those  found  in  the  1993-94  field  performance  study  of  the  older  version 
of  COBRA3.  The  COBRA  II  overall  mean  RMS  reproducibility  value  was  found  to  be 
8.64%,  while  for  COBRA  it  was  calculated  to  be  9.71%,  but  is  not  listed  in  the  previous 
report.5  By  this  overall  measure,  COBRA  II  performed  1 1%  better  in  reproducibility  than 
COBRA.  The  repeatability  and  reproducibility  values  for  each  base  lab  are  shown  in 
Figures  3  and  4,  respectively,  with  95%  confidence  intervals  illustrated. 

All  lab  repeatability  values  were  less  than  6%,  while  all  lab  reproducibility  values 
were  less  than  15%,  absolute.  All  but  2  labs  had  less  than  10%  normalized  difference 
from  the  overall  RMS  reproducibility  value.  The  95%  confidence  interval  for 
repeatability  is  approximately  2-5%,  meaning  that  a  single  instrument  will  usually  have 
less  than  5%  RMS  variability  when  analyzing  the  same  sample.  For  a  worst  case 
analysis,  the  99%  confidence  interval  for  repeatability  shows  that  RMS  variability  from  a 
single  COBRA  II  should  almost  always  be  less  than  7%.  The  worst  lab  repeatability 
value  in  this  study  was  5.74%.  The  95%  confidence  interval  for  reproducibility  is  about 
-6  to  6%.  This  means,  in  absolute  terms,  that  one  can  normally  expect  less  than  12% 

RMS  variation  in  readings  from  instrument  to  instrument.  The  99%  confidence  interval 
for  reproducibility  is  about  -15%  to  15%.  Therefore,  one  could  get  an  RMS  variability 
from  one  COBRA  II  to  another  of  up  to  30%,  in  rare  instances.  In  this  study,  the  total 
reproducibility  value  range  was  -14.7  to  11.9%,  or  about  27%. 


Comparison  with  JOAP  Spectrometer  Pata 

It  is  also  informative  to  compare  the  COBRA  II  data  with  typical  performance 
data  from  atomic  emission  wear  metal  analysis  spectrometers,  which  are  used  in  routine 
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Figure  3.  RMS  Repeatability  Values  for  all  Base  Labs 

oil  analysis  at  many  bases  as  part  of  the  DOD’s  Joint  Oil  Analysis  Program  (JOAP).  It 
should  be  noted  that  the  comparison  here  is  only  with  respect  to  numerical  data.  The 
JOAP  spectrometers  and  COBRA  H  measure  completely  different  types  of  oil  properties. 
The  COBRA  II  measures  the  relative  electrical  conductivity  of  oils,  while  the  JOAP 
spectrometer  measures  the  concentration  of  wear  metal  contaminants  contained  within  the 
oil.  The  JOAP  atomic  emission  spectrometer  has  capability  to  detect  the  presence  of  20 
different  metals,  with  varying  degrees  of  sensitivity. 

For  comparison  of  COBRA  H  repeatability  with  that  of  the  JOAP  spectrometer, 
specification  data  from  a  typical  JOAP  instrument  were  examined.  Table  3  gives  the 
repeatability  specifications  (allowable  standard  deviations)  for  a  typical  JOAP  atomic 
emission  spectrometer  . 
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Figure  4.  RMS  Reproducibility  Values  for  all  Base  Labs 

At  a  JOAP  reading  level  of  3  parts  per  million  (ppm),  the  repeatability 
specification  has  a  range  of +0.52  to  1.02  ppm  or  about  17-34%  RSD,  depending  on 
element.  This  reading  level  is  comparable  to  the  COBRA  II  mean  of  2.98  for  sample  1 . 
The  average  COBRA  II  standard  deviation  for  sample  1  was  ±0.17  or  about  6%,  while 
the  largest  was  ±0.55  (18%).  The  18%  RSD  is  very  close  to  the  tightest  JOAP 
spectrometer  specification  at  this  level. 

At  a  JOAP  spectrometer  reading  level  of  10  ppm,  roughly  corresponding  to  the 
COBRA  U  mean  of  8.47  for  sample  2,  the  repeatability  specifications  range  from  ±0.71 
to  1 .30  ppm  or  about  7-13%.  The  average  standard  deviation  for  COBRA  n  at  this 
reading  level  was  ±0.25  (3%).  The  largest  standard  deviation  was  ±0.71  (8%), 
approximately  equal  to  the  lowest  JOAP  specification  limit  at  this  level. 

Linear  interpolation  of  Table  3  between  10  and  20  ppm  concentrations,  gives  a 
repeatability  specification  range  of  approximately  ±0.92  to  1.88  ppm  or  about  6-13%  at  a 
reading  of  15  ppm.  This  reading  level  corresponds  to  the  mean  of  15.2  for  sample  3. 


14 


TABLE  3.  Repeatability  Specifications  (Allowable  Standard  Deviations)  for  a 
JOAP  Atomic  Emission  Spectrometer7 

REPEATABILITY  SPECIFICATIONS 


The  average  standard  deviation  for  sample  3  was  ±0.36  (2%),  while  the  largest  was  ±0.71 
(5%),  which  is  below  the  interpolated  JOAP  spectrometer  specification  range  at  this  level. 

Based  on  this  comparison,  the  repeatability  of  the  COBRA  II  instruments  tested  in 
this  study  appears  to  be  quite  acceptable  with  respect  to  criteria  long  accepted  for  JOAP 
atomic  emission  spectrometers. 

For  a  comparison  of  reproducibility  of  COBRA  II  with  the  JOAP  spectrometer, 
one  year’s  data  from  the  JOAP  monthly  correlation  reports8  were  examined.  To  simplify 
the  analysis,  only  data  for  the  element  iron  (Fe)  were  evaluated.  Iron  is  the  most 
important  wear  metal  analyzed  by  the  JOAP  spectrometer,  and  is  also  among  the  most 
repeatable  and  reproducible  elements  detected  by  atomic  emission.  The  correlation 
program  values  were  calculated  from  the  overall  true  (non-trimmed)  means  and  standard 
deviations  for  4  JOAP  samples  analyzed  by  approximately  200  labs  each  month.  The 
overall  monthly  RMS  reproducibilities  for  iron  over  a  one  year  period  ranged  from  6.0- 
1 3%.  These  values  were  calculated  exactly  as  for  the  COBRA  II  overall  mean  RMS 
reproducibility  value,  R*,,  found  here  to  be  8.64%.  The  yearly  average  for  the  JOAP 
spectrometer  was  8.62%,  statistically  identical  to  the  COBRA  II  value. 
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The  overall  RMS  reproducibilities  of  COBRA  II  and  the  JOAP  spectrometer 
appear  to  be  very  similar.  This  should  be  a  statistically  valid  observation,  since  the 
COBRA  II  database  is  significant  beyond  the  99%  confidence  level  and  the  JOAP  data 
base  is  so  huge  as  to  be  considered  absolute.  COBRA  II  data  appear  to  be  reliable  to 
approximately  the  same  extent  as  those  from  the  JOAP  atomic  emission  spectrometer. 

Field  Performance  Summary 

As  of  September  1997,  nine  black  oil  problems  were  successfully  detected  by 
COBRA  II  during  the  field  service  evaluation  CIP6.  However,  there  were  also  three  false 
calls  and  two  precautionary  engine  removals  where  no  evidence  of  coking  or  distress 
were  found.  Also,  one  engine  was  pulled  for  scheduled  LPT  and  core  time  change  and 
found  to  have  internal  and  external  coking  of  the  No.  5  pigtail  tube  with  a  COBRA  II 
reading  of  4,  but  no  trend  data  was  available  (this  was  first  COBRA  II  reading  on  the 
engine).  Table  4  details  the  findings  for  the  15  engine  removals6. 


Table  4.  Engine  Removal  Findings  for  CIP  Task  3206  (COBRA  II  Field  Service 
Evaluation) 


Date 

Engine 

Base 

Reason  for  Removal 

Results  of  Teardown 

Conclusion 

4/23/97 

680874 

Eglin 

1  or  2  to  5  increase 

Ext./lnt.  coke  on  No.  5  Tubes 

Save 

4/11/97 

680644 

Elmendorf 

7  to  20  one  flight  increase 

No.  5  Oil  Supply  Tube  coked 
internally 

Save 

3/3/97 

719167 

Seymour 

Johnson 

1  to  4  increase 

No.  5  Tubes  coked  internally 

No.  5  Adapter  Jet  clogged 

Save 

2/13/97 

681584 

Elmendorf 

LPT  and  core  time  change 
COBRA  =  4 

Ext./lnt.  coke  on  No.  5  Pigtail 
Tube.  No  Cobra  trend  data. 

None 

10/24/96 

712037 

Mt.  Home 

COBRA =  10 

No.  5  Adapter  Jet  clogged 

Save 

6  97 

719039 

Mt.  Home 

High  COBRA 

No  hardware  problems.  Oil 
contaminated  by  water 

False  Call 

5/21/97 

719154 

Mt.  Home 

1  to  5  one  flight  increase 

Evidence  of  bore  fire.  No.  5 
Adapter  Jet  plugged 

Save 

5/9/97 

712284 

Eglin 

3  to  5  jump 

No  problems  found  in  No.  5 
Adapter  Jet  and  Scavenge  Tube. 
Engine  RTS 

Precautionary 

Pull 

5/8/97 

719263 

Seymour 

Johnson 

1  to  3  jump 

Chunks  of  coke  in  the  #5  Adapter  Save 

Jet,  Pressure  and  Transfer  Tube 

8/21/97 

720135 

Elmendorf 

"4"  to  "6"  readings,  dropped 
to  "2",  returned  to  "5" 

No  findings 

False  call 
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Date 

7/25/97 

Engine 

720076 

Base 

Elmendorf 

Reason  for  Removal 

Jump  3  to  6  in  3  hrs. 

Results  of  Teardown 

No.  4  Rear  Seal  Seat  oil 
passages  plugged 

Conclusion 

Save 

7/8/97 

719306 

Lakenheath 

COBRA  =10 

No.  5  Compartment  coked 

Save 

5/21/97 

682284 

Eglin 

3  to  5  jump 

All  hardware  clean 

Precautionary 

Pull 

9/19/97 

720137 

Elmendorf 

"4"  avg.  reading,  dropped  to 
"1",  returned  to  "5" 

All  hardware  clean 

False  call 

N/A 

711823 

Eglin 

1  to  3  increase 

Pigtail  Tube  showed  evidence  of 

Save 

leakage.  Small  coke  deposits  in 
the  No.  5  Adapter  Jet 


Of  the  3  false  calls,  one  was  due  to  an  abnormally  high  concentration  of  water  in 
the  oil,  according  to  the  field  report  from  Mountain  Home  AFB  ED.  Increased  water 
concentration  leads  to  increased  conductivity  and,  therefore,  higher  COBRA  II  readings. 
The  other  two  false  calls  occurred  when  engines  were  pulled  at  Elmendorf  AFB  AK,  due 
to  COBRA  II  readings  that  dropped  by  3  or  4,  then  rose  again  to  the  previous  level.  The 
reason  for  the  unusual  drop  in  readings  has  not  been  found.  There  were  records  of 
component  changes  on  these  engines  at  the  time  of  the  decreased  readings,  but  no  report 
of  whether  the  oil  had  been  changed  or  not9.  An  oil  change  or  significant  addition  of  new 
oil  could  cause  such  a  decrease  in  reading.  If  significant  new  oil  addition  was  not  the 
culprit,  then  the  COBRA  II  instrument  at  Elmendorf  may  not  be  operating  properly. 
However,  personnel  there  say  it  appears  to  be  doing  fine,  with  several  thousand  “normal” 
readings  and  3  saves  so  far. 

At  no  time  has  COBRA  II  yet  failed  to  detect  a  problem,  when  one  actually 
existed.  There  were  over  14,000  COBRA  II  analyses  from  523  engines  reviewed  during 
the  CIP.  Over  99.4%  of  the  readings  were  3  or  less.  There  were  three  precautionary 
engine  removals  when  readings  jumped  by  2  from  the  previous  flight.  No  coking  or 
hardware  problems  were  found  in  two  of  these  pulls,  where  the  COBRA  II  readings 
jumped  from  3  to  5.  However,  chunks  of  coke  were  found  in  the  No.  5  adapter  jet  in  one 
of  the  precautionarily  pulled  engines,  when  the  reading  jumped  from  1  to  3.  The  CEP 
team  has  come  up  with  the  following  lessons  learned  and  recommendations6: 

LESSONS  LEARNED: 

1 .  Water  contamination  causes  false  high  readings. 

2.  COBRA  II  “jump”  can  be  caused  by  aberrant  low  reading  followed  by  return 
to  normal  trend. 

3.  Jump  of  2  is  usually  not  significant. 
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RECOMMENDATIONS : 


1.  Reject  engine  for  COBRA  II  reading  of  10  or  above. 

2.  Reject  engine  if  COBRA  II  reading  increases  3  or  more  above  the  average 
reading  for  the  last  10  operating  hours. 

3.  Drain  and  flush  engine  if  there  is  reason  to  suspect  water  contamination  (look 
for  “cloudy”  or  “milky”  appearance  of  oil  sample;  do  crackle  test  or  run  Karl  Fischer 
titration  and  drain/flush  if  above  1,500  ppm  H20). 

4.  Investigate  if  COBRA  II  reading  decreases  from  previous  reading. 

Fifty  COBRA  II  instruments  have  been  purchased  to  date  at  a  total  cost  of  $22  5K. 
Nine  engine  saves  have  been  documented  so  far  during  the  CIP.  Additionally,  COBRA  II 
is  responsible  for  at  least  4  other  F100  engine  saves  prior  to  the  CIP.  Therefore,  savings 
to  the  FI 00  program  are  at  least  $39M,  based  on  an  estimated  $3M  for  each  FI 00  engine. 
This  is  a  return  on  investment  (ROI)  of  over  17,000%!  COBRA  II  is  having  an  extremely 
positive  impact  on  the  FI 00  maintenance  program  and  should  remain  in  the  field  until 
such  time  as  the  current  No.  5  bearing  compartment  retrofit  is  completed  and  proven  to 
alleviate  this  engine’s  black  oil  problem. 
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5.  CONCLUSIONS 


COBRA  II  data  are  found  to  be  repeatable  and  reproducible  within  a  95% 
confidence  interval  of  less  than  5%  and  12%  RMS,  respectively.  The  newer  COBRA  II 
shows  approximately  10-15%  better  performance  in  these  parameters  than  the  older 
version  of  COBRA.  The  repeatability  and  reproducibility  levels  for  COBRA  II  are  found 
to  be  acceptable  and  very  similar  to  the  performance  criteria  for  data  from  the  widely 
used  JOAP  atomic  emission  spectrometer. 

The  field  performance  of  COBRA  II  has  been  very  good  so  far,  with  at  least  13 
engine  saves.  Savings  to  the  FI 00  engine  maintenance  program  attributable  to  use  of 
COBRA  II  are  estimated  to  be  at  least  $39M,  with  a  return  on  investment  of  over 
17,000%.  The  COBRA  II  is  a  cost-effective  and  safety  enhancing  instrument,  when  used 
in  turbine  engine  maintenance  programs  where  significant  oil  thermal  degradation  or 
coking  problems  exist. 
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COBRA  II  Summary  Statistics  and  Raw  Data 
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Summary  Statistics 


09/30/97  03 : 10 : 59  PM 

Variable:  Samplel 

Sample2 

Sample3 

Sample  size 

45 

45 

45 

Average 

2.97778 

8.46667 

15.1778 

Median 

3 

9 

15 

Mode 

3 

9 

15 

Geometric  mean 

2.95757 

8.43027 

15.1256 

Variance 

0.113131 

0.618182 

1.60404 

Standard  deviation 

0.33635 

0.786245 

1.26651 

Standard  error 

0.0501401 

0.117207 

0.1888 

Minimum 

2 

7 

13 

Maximum 

4 

10 

18 

Range 

2 

3 

5 

Lower  quart ile 

3 

8 

15 

Upper  quart ile 

3 

9 

16 

Interquartile  range 

0 

1 

1 

Skewness 

-0.41732 

-0.178775 

-0.0690084 

Standardized  skewness 

-1.14288 

-0.489595 

-0 . 188987 

Kurtosis 

6.80986 

-0.343161 

-0.393639 

Standardized  kurtosis 

9.32478 

-0.469893 

-0.539013 

Coeff.  of  variation 

11.2953 

9.28636 

8.34448 

Sum 

134 

381 

683 

—  frPt  CWF  JNWtVAL: 

x.  fi-i.ix 

7.iS-?.X6 

n ,°i  -  us 
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Raw  Data 


FILE:  COBRACOR  10/01/97  09:09:32  AM 


Row 

Location 

Samplel 

Sample2 

Sample 3 

1 

Seymour  Johnson 

3. 

7. 

13. 

2 

Seymour  Johnson 

3. 

7. 

13  . 

3 

Seymour  Johnson 

3. 

7. 

13  . 

4 

Seymour  Johnson 

3. 

7. 

13  . 

5 

Seymour  Johnson 

3. 

7. 

13  . 

6 

Elmendorf 

3. 

10. 

15. 

7 

Elmendorf 

4  . 

8. 

16. 

8 

Elmendorf 

4. 

9. 

15. 

9 

Elmendorf 

3. 

9. 

16. 

10 

Elmendorf 

3. 

9. 

16. 

11 

Nellisl 

3. 

9. 

16. 

12 

Nellisl 

3. 

9. 

16. 

13 

Nellisl 

3. 

9. 

17. 

14 

Nellisl 

3. 

9. 

16. 

15 

Nellisl 

3. 

9. 

17. 

16 

Nellis2 

3. 

8. 

15. 

17 

Nellis2 

3. 

8. 

15. 

18 

Nellis2 

3. 

8. 

15. 

19 

Nellis2 

3. 

9. 

15. 

20 

Nellis2 

2  . 

8. 

15. 

21 

Wright -Patterson 

3. 

9. 

15  . 

22 

Wright -Patterson 

3. 

9. 

15. 

23 

Wright -Patterson 

3. 

9. 

15. 

24 

Wright - Patterson 

3. 

9. 

15. 

25 

Wright -Patterson 

3. 

9. 

15. 

26 

Eglin 

2. 

9. 

15. 

27 

Eglin 

2. 

9. 

15. 

28 

Eglin 

3. 

8. 

16. 

29 

Eglin 

3. 

8. 

16. 

30 

Eglin 

3. 

9. 

16  . 

31 

Langley 1 

3. 

8. 

14. 

32 

Langleyl 

3. 

8. 

13  . 

33 

Langleyl 

3. 

8. 

14. 

34 

Langleyl 

3. 

8. 

14  . 

35 

Langleyl 

3. 

8. 

14. 

36 

Langley2 

3. 

8. 

16. 

37 

Langley2 

3. 

8. 

15. 

38 

Langley2 

3. 

8. 

15. 

39 

Langley2 

3. 

8. 

14. 

40 

Langley2 

3. 

8. 

15. 

41 

Luke 

3. 

9. 

17. 

42 

Luke 

3. 

9. 

17. 

43 

Luke 

3. 

9. 

18  . 

44 

Luke 

3. 

10. 

17. 

45 

Luke 

3. 

10. 

17. 
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Frequency  Hietogrem 


Semple  1 


1 
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3 
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COBRA  IX  Reading 
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Frequency  Hietogrem 


Semple  2 
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COBRA  II  Reeding 
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Threa-D  Histogram 


COBRA  II  RMS  Lab  Valuas 
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